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Abstract 

T his paper presents a conceptual framework for 

understanding the uses of research in policy and practice, 
findings from recent empirical work, and early lessons from 
the field. The framework describes the ways policymakers 
and practitioners define, acquire, interpret, and ultimately 
use research. Relationships are vital conduits for acquiring 
research. When confronted with questions about a program or reform, 
agencies and legislators often turn to trusted peers and intermediaries. 
Translation is also key. Because research does not speak for itself, 
policymakers and practitioners must always interpret its meaning and 
implications for their particular problems and circumstances. This means 
that identifying the right translators and creating productive conditions 
for translation are critical. Policymakers and practitioners use research 
in various ways, including instrumental, conceptual, political, imposed, 
and process uses. Increased knowledge of these nuances should enable 
researchers to produce more useful work and better engage with 
policymakers, practitioners, and intermediaries. 
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From the Editors 

Since Francis Bacon established the scientific method, there have surely 
been individuals trying to figure out how the knowledge generated by science 
could be applied and indeed be used in the “real world.” The current inter- 
est in evidence-based practice in nearly all fields of human service is based 
on the premise that the human service will be optimized, and/or advanced 
by employing practices that have scientific evidence of effectiveness. The 
field of Education is an important exemplar of this trend, and the success of 
Education in promoting the development and learning of children and youth is 
of primary concern for the United States and most other nations. Yet, the gap 
between research and practice in this country is wide. 

In this Social Policy Report, Dr. Tseng examines the role of research in 
policy and practices in Education. She begins by asking rhetorically why one 
should conduct research on the use of research and provides a rationale for 
it being more that a self-reflective activity. Following an ecological systems 
conceptualization of research and factors affecting its use, Dr. Tseng discusses 
the different definitions of “research” by consumers (e.g., policymakers and 
practitioners) and educational researchers and factors mitigating its use in 
practice. She then describes a program of research funded by the W. T. Grant 
Foundation to build “stronger theory and empirical evidence on how, when, 
and under what conditions” research will be used. Building on the concepts 
of knowledge utilization, implementation science, and translational research, 
the projects funded by W. T. Grant address the practical, process questions 
that are rarely addressed in Randomized Control Trials (RCTs) and even “scal- 
ing up” studies. Dr. Tseng describes three exemplary projects that are now 
studying ways in which school boards, school staff, and social welfare agen- 
cies view, understand, and use research. From this research and the broader 
literature, she suggests early lessons learned from this research and their 
implications for future research. 

Commenting on this paper, Dr. Aletha Huston notes the different cultures 
within which researchers and practitioners are situated, the importance of 
the similarities between the contexts of where research is conducted and 
where it will be used, and the threat of evaluation that some practitioners 
field from research because of the increased emphasis on accountability. 
Similarly, Dr. John Easton comments on the importance of grounding edu- 
cation research in practice occurring in the field, responding to the needs 
expressed by school districts and practitioners, and establishing partnerships 
between researchers and school districts. From a state policy perspective, Dr. 
Karen Cadigan speaks about the distant worlds of policymakers and research- 
ers and a need for a common mode of communication between the two. She 
emphasizes the importance of researchers being able to respond to policy- 
makers’ information needs in a timely (i.e. , quick) manner and with an actual 
answer, which has never been a strength of academia. 

In sum, Dr. Tseng provides a critically important set of perspectives and 
insights on factors affecting use of research in educational practice. Her sup- 
position that increasing scientific rigor is not enough to impact significantly 
the Education enterprise in the U.S. should be taken to heart. The next best 
hope may well be the knowledge utilization/ implementation science work 
(that builds on rigorous science) now being conducted. 

-Samuel L. Odom (Lead Editor) 
Donna Bryant (Editor) 
Kelly L. Maxwell (Editor) 
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The Uses of Research in 
Policy and Practice 


T he call for evidence-based policy and prac- 
tice is nearly ubiquitous. It can be heard 
across the fields of education, child wel- 
fare, mental health, juvenile justice, youth 
programs, and health care. On the research 
side, billions of dollars are spent on efforts 
to generate stronger research evidence; while on the 
policy and practice sides, higher stakes and incentives are 
attached to the use of research evidence. Despite these 
valiant efforts, critical gaps remain between research, 
policy, and practice. These gaps are exacerbated by a lack 
of knowledge about how researchers can produce more 
useful work, how practitioners can acquire and use that 
work productively, and how policymakers can create the 
conditions that enable both to occur. Unless this changes, 
it seems likely that the hope for evidence-based policy and 
practice will unravel-another fad tried and failed. 

The research community needs a stronger under- 
standing of how practitioners and policymakers engage 
research. This understanding should include their defini- 
tions of research, their perceptions of its relevance and 
quality, their preferred modes of communication, and 
the forces that influence their use of research. Aletha 
Huston, in her 2005 Social Policy Report and 2008 SRCD 
Presidential Address, calls for more useable research. 
Nancy Guerra, Sandra Graham, and Patrick Tolan (2011) 
similarly argue for more use-inspired research in their 
Child Development special issue. These scholars also 
argue for stronger communication of research findings. 
Without a strong understanding of the worlds on “the 
other side” of the gaps, however, scholars’ efforts to 
make research more useful and to communicate it more 
effectively run the risk of missing the mark. 

Understanding how practitioners and policymakers 
use research is an area that is ripe for scientific study. 
This paper discusses the importance of studying the use 
of research, presents a conceptual framework for under- 
standing the issue, and draws lessons from recent work? 


Why Study the Use of Research? 

The study of research use has its roots in the 1970s and 
1980s, a time Henry and Mark (2003) called the “golden 
age” for work on evaluation use and knowledge utili- 
zation. Carol Weiss, a leading figure in this field, was 
initially motivated to understand why government would 
support research but not use the findings: 

“I was asked to evaluate a program of the 
‘War on Poverty’ in the 1960s. Lyndon John- 
son’s policy to ‘eradicate poverty’ generated 
a whole range of new programs: education, 
health, mental health, job training, programs 
for the elderly and so on.... When I finished 
my evaluation of the Harlem program, the 
report came out in three volumes. We sent 
copies of the report to Washington: I never 
heard a word from them! I had the feeling I 
could have just dumped it into the ocean and 
it would have made no difference. So, I asked 
myself: ‘Why did they support and fund this 
evaluation if they were not going to pay any 
attention to it?’ That’s how I got interested 
in the uses of research: What was going on? 
What could researchers-or anyone else-do 
to encourage people to pay more attention to 
research?” (in Graff & Christou, 2001). 

Michael McPherson, president of the Spencer 
Foundation, which funds education research to inform 
practice, recently offered this perspective: 

“This problem of how research becomes ef- 
fective in practice is itself a social science 
question of considerable depth and complex- 
ity that deserves study in its own right. The 
paths by which research knowledge finds its 
way into the daily life of educational orga- 
nizations, the paths by which practitioner 
knowledge is brought to bear and made to 
count in the research process, and the paths 
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by which researchers become interested in 
problems of genuine importance to practice 
are complex and hard to understand and 
warrant systematic analysis and reflection” 
(McPherson, 2004, p. 8-9). 

These challenges are familiar to many researchers. 
At the William T. Grant Foundation, we have a longstand- 
ing interest in supporting research that can inform policy 
and practice to improve the lives of young people. We 
support research on after-school programs, child welfare, 
education, mentoring, juvenile justice, and child and 
adolescent mental health. On notable occasions this work 
has influenced practice or policy, but that has not been 
the norm. Like others in the field, we have strong ideas 
about why research is influential in some instances but 
not others. These ideas often stem from 20/20 hind- 
sight about past situations in which research seemed 
influential, but they do not constitute a strong explana- 
tory framework that has been tested and helps predict 
future uses of research. A few years ago, the Foundation 
launched an initiative to build strong theory and empiri- 
cal evidence on when, how, and under what conditions 
research is used. Our ultimate goal is to understand how 
to develop better research, improve its use, and build 
stronger bridges across those notorious gaps between 
research and policy, and research and practice. 


Limitations in Current Approaches 

People often describe the need to move “research to 
practice” or “research to policy.” Left there, these ap- 
proaches can seem like a one-way street, neglecting 
the equally important need to move an understanding 
of practice and policy to research. Existing approaches 
focus more often on practice than policy, and reflect 
what Nutley, Walter, and Davies (2007) call producer-push 
models. The underlying logic of these approaches is that 
researchers should produce high-quality research, make 
it clear and accessible, and then practitioners should ap- 
ply it to their work. 

Considerable effort has focused on the first step 
in the model: improving the quality of research. Most 
of these efforts have sought to improve scientific rigor 
in testing “what works” questions about the effective- 
ness of programs and practices and in synthesizing 
that evidence. For example, the Society for Prevention 
Research created a Standards of Evidence Committee 
that developed criteria for efficacy, effectiveness, and 
dissemination studies. Several organizations have defined 
standards of evidence (and syntheses of it) to create lists 


of effective programs; these include the Department of 
Education’s What Works Clearinghouse, Substance Abuse 
and Mental Health Services Administration’s National 
Registry of Evidence-based Programs and Practices, the 
California Clearinghouse for Evidence-Based Practice in 
Child Welfare, and the Office of Juvenile Justice and 
Delinquency Prevention-supported Blueprints for Violence 
Prevention. To a lesser extent, the research community 
has developed scientific standards in additional areas, 
such as the National Research Council’s Committee on 
Scientific Principles for Education Research (2002) and 
the National Science Foundation’s (2003) standards for 
qualitative research. 

Efforts have also focused on the second part of the 
producer-push model: improving communication, dis- 
semination, and marketing of research. These efforts are 
intended to provide research to end-consumers in more 
accessible forms (Nutley, et al., 2007). The most com- 
mon strategy involves presenting research in written and 
verbal formats that are suitable for policy and practice 
audiences. Policy briefs and executive summaries are 
designed to be short and jargon-free so that they can be 
quickly read by busy people who do not have research 
training. Briefings sometimes supplement the dissemina- 
tion effort. Websites and searchable databases represent 
another approach, wherein research is centrally stored 
and made broadly available. Others take a marketing ap- 
proach, “packaging” research in more attractive ways so 
that it can compete in the marketplace of glossy prod- 
ucts sold to practitioners. 

A third body of work in the vein of research-to- 
practice is variously referred to as dissemination, imple- 
mentation, scale-up, and translational research (Centers 
for Disease Control and Prevention /Office of Public Health 
Research, 2002; Flay et al., 2005; National Institute on 
Drug Abuse, 2001; National Institute of Mental Health, 
2002; National Research Council, 2009). This work draws 
on medical research models and resembles the phases of 
clinical drug trials. The prevention research model, which 
is based on those medical models, begins with descriptive 
research to understand the risk and protective factors 
that contribute to particular outcomes; uses those find- 
ings to develop a program to improve outcomes; and then 
tests the efficacy, effectiveness, and dissemination and/or 
implementation of the program (National Research Coun- 
cil, 2009). More recent iterations of the prevention model 
include feedback loops, in which what is tried in practice 
informs future research, but the predominant flow is still 
one of moving research to practice. 
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Connecting research and practice should be more of 
a two-way street than is implied in research-to-practice 
approaches. Without a concomitant focus on how prac- 
tice should inform research, we risk privileging research- 
ers’ perspectives and relegating practice professionals to 
the receiving end of research and dissemination efforts. 
Moreover the producer-push models may help, but they 
have achieved limited success. 

A more significant shift is needed to strengthen the 
connections between research and practice as well as 
research and policy. As a field, we have spent much of our 
energy on the supply side (Nutley, et al. , 2007; Porter, 
2007; Tseng, 2009), and we need to add a stronger focus 
on demand. Without this shift, we have little systematic 
understanding of whether our supply-side attempts ad- 
dress demand-side needs, nor how to improve our efforts. 

A Conceptual Framework for 
Understanding the Uses of Research 

In focusing on the demand side, the intended research us- 
ers become the center of our inquiries. Below I describe 
a conceptual framework for identifying research users; 
examining how they define, acquire, interpret, and use 
research; and understanding the social ecology that influ- 
ences those processes. 


Who are Research Users? 

An important starting place is identifying the intended 
research users. Too often, the research community 
makes broad-based calls for research to influence policy 
or practice without specifying the decision-makers they 
hope to reach. Up to this point, I have been guilty of us- 
ing the handy but faulty grouping of policymakers versus 
practitioners, as if there are two distinct groups that can 
be clearly delineated. In reality, the landscape of actors 
and organizations is more nuanced and so is the range of 
potential research users. 

One strategy for identifying research users is to 
consider the decisions or issues researchers seek to in- 
form, and then work backward to the organizations and 
decision-makers who play key roles. Child development 
researchers often seek to inform federal policymakers 
and frontline practitioners, but miss the mid-level actors 
and organizations that are better positioned to draw on 
research to shape youth’s daily settings and lives. State 
and local agencies are a critical group of research us- 
ers. This includes state and local departments of social 
services, mental health, education, juvenile justice, and 


employment. Within these agencies, mid-level admin- 
istrators and program managers shape the frontline 
practices— teaching, social work, counseling, policing-of 
concern to many researchers. They play a critical role in 
designing staff development systems and adopting new 
programs and reforms, shaping the process and conditions 
for their implementation, and allocating resources in sup- 
port of them. These mid-level decision makers straddle 
policy and practice and are well-poised to put research 
to work to benefit youth. In addition, they can be a more 
stable presence than agency leaders, who have short 
tenures in many places. 

These state and local agencies currently face 
unprecedented demands from politicians, courts, and 
funders to use research and data. The No Child Left 
Behind Act (2002), for example, included more than 100 
references to “scientifically based research” and required 
that school districts use research in their decisions about 
curricula, instructional programs, and professional devel- 
opment. The Obama administration has made program 
evaluation a priority across agencies, and wants to use 
evaluations to “help the Administration determine how 
to spend taxpayer dollars effectively and efficiently- 
investing more in what works and less in what does not” 
(Orszag, 2009). Recent legislative and judicial actions in 
some states have also required that child welfare and 
mental health agencies use research to redesign sys- 
tems, select evidence-based programs and practices, and 
implement them. 

Another critical group of research users is interme- 
diaries. They are the organizations and individuals who 
translate and package research for use by legislators, 
agency staff, and nonprofit and private service providers. 
They also broker relationships between researchers and 
practitioners/ policymakers. The number of intermediar- 
ies has proliferated in recent decades. Advocacy groups 
and think tanks play an important role in determining 
which policy ideas gain ground, which reform efforts 
fail, and how money gets appropriated. State and local 
agencies lacking the capacity to draw on research rely on 
professional associations, technical assistance providers, 
and consultants for research. There is considerable vari- 
ability across intermediaries in their research expertise, 
but there is little denying their significant roles in lever- 
aging research to shape policy and practice. 


How is Research Defined, Acquired, Interpreted, and Used? 

The implicit model for how research is acquired, inter- 
preted, and used is a rational and linear one. It is easy 
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to imagine a decision-maker who encounters a dilemma, 
goes out in search of information to address the question 
at hand, finds research that provides the missing infor- 
mation, and uses it to decide. Unfortunately, this image 
rarely bears out in reality. Those who study research use 
find more complicated processes at work. 

Defining Research. Although the terms research, 
evidence, evidence-based policy, and evidence-based 
practice are frequently bandied about, people in differ- 
ent communities often hold different definitions of these 
terms. Three decades ago, Nathan Caplan (1979) argued 
that language differences between researchers and poli- 
cymakers obstructed the use of research; scholars have 
since extended his analysis to research-practice gaps. At 
the most basic level, the 
language gap begins with 
how researchers, practi- 
tioners, and policymakers 
define research and evi- 
dence. What should qualify 
as research and evidence 
for use is a hotly contested 
topic, and it is not my goal 
to engage in that debate 
here. Regardless of what 
people think “should” 
qualify as research, it is im- 
portant to understand what 
people actually think it is. 

Recognizing definitional 
differences helps people 
avoid inadvertently talk- 
ing past one another. More 
significantly, knowing what 
people in different roles 
believe constitutes research, and why they hold those 
views, is important for moving toward shared understand- 
ing about the uses of research. 

Researchers often employ the terms evidence 
and research interchangeably, defining them as empiri- 
cal findings derived from scientific methods. Studies of 
research use in education suggest that policymakers and 
practitioners have broader definitions. In their review of 
more than 50 studies, Honig, and Coburn (2008) found 
that school district staff drew on a wide array of evi- 
dence, which encompassed social science research, as 
well as student achievement data, expert testimony, 
practitioner knowledge, and parent and community 
input. Nelson, Leffler and Hanson (2009) interviewed 


congressional education staffers and conducted focus 
groups with chief state school officers, state legislators, 
superintendents, curriculum coordinators, and school 
board members. Those policymakers and practitioners 
also employed a broad conceptualization of research that 
included empirical findings, data, personal experiences 
and the experiences of others, and constituent feedback. 
Yet another definition is the one mandated by law. The 
No Child Left Behind Act (2002, subpart 37 of section 
9101 ) employs the term scientifically based research, 
defining it as “research that involves the application of 
rigorous, systematic, and objective procedures to obtain 
reliable and valid knowledge relevant to education activi- 
ties and programs.” This definition is inclusive of research 

using “observational or 
experimental methods,” 
but those interviewed in 
Nelson’s study often associ- 
ated the term with “gold 
standard” randomized 
controlled trials. 

Research also comes 
in different shapes and 
forms. Most people would 
agree that research in- 
cludes findings from a 
single study or syntheses 
of findings from multiple 
studies. For many people, 
research is also embodied 
within products, such as 
practice guidelines, cur- 
ricula, evidence-based 
programs, and assessment 
tools. Researchers often 
think of these products as research-based if they have 
been studied, were developed using research findings, 
and/or were part of a research and development process. 
These products represent an interesting case because 
practitioners do not need to review the empirical work, 
nor know of their research basis when they use them. 

Acquiring Research. Researchers have focused 
a great deal on how to better push information out, 
but they have not developed strong knowledge of how 
practitioners or policymakers pull information in (Dearing 
& Kreuter, 2010). We have sought to better disseminate 
research by presenting it in clearer written and verbal 
formats and making it more readily accessible through 
online sources (Nutley, et al. , 2007). What we lack is a 


Researchers often employ the 
terms evidence and research 
interchangeably, defining them 
as empirical findings derived 
from scientific methods. Studies 
of research use in education 
suggest that policymakers and 
practitioners have 
broader definitions. 
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deep understanding of how the intended users typically 
acquire research and other types of information. Shifting 
our vantage point to the user side causes us to ask differ- 
ent questions-what are their main sources of informa- 
tion, how do they come to trust certain sources over oth- 
ers, how do they seek out new information, and how does 
it routinely come into their hands? As I discuss below, we 
are only beginning to understand the answers to such 
questions, but early work suggests that it is important to 
consider policymakers’ and practitioners’ social systems— 
sets of relationships and interactions with people they 
trust and who are trying to solve similar problems. 

Interpreting Research. Research does not speak for 
itself, nor does it have definitive implications for particu- 
lar problems of practice or policy. Research users must 
always interpret the meaning of research and its implica- 
tions for their specific problems and decisions. Knowl- 
edge from research does not remain as distinct pieces of 
information once it enters people’s minds and discourse. 
As Kennedy (1984, p. 225) puts it, “evidence is not merely 
attached to the user’s store of knowledge like barnacles 
are to clams... rather [it] is a formative process in which 
evidence is acted on by the user. It is sorted, sifted, 
and interpreted; it is transformed into implications and 
translated into inferences.” The would-be users are also 
assessing the quality and credibility of research, and not 
necessarily doing it with the same criteria researchers 
use. The ways each group appraises research and deems 
it credible is based on their professional norms and train- 
ing, their prior knowledge, their goals for and rules of 
evidence, and whether the research is actionable or chal- 
lenges the status quo (Bogenschneider & Corbett, 2010; 
Caplan, 1979; Shonkoff, 2000; Weiss & Bucuvalas, 1980). 

Using Research. There are also various ways of 
using research (see summary by Nutley, et al. , 2007). 
Researchers often hope for instrumental use, wherein 
research directly influences a policy or practice decision. 

It is the rational and linear image of research use wherein 
a decision-maker has a question and uses research to 
address that question and make a decision. The research 
community often bemoans political uses, in which re- 
search is used to justify a position that has already been 
staked out. In this case, policymakers or practitioners 
know whether they support or oppose a piece of legisla- 
tion or reform effort, and they marshal research to back 
their position. This type of use is also referred to as tacti- 
cal or symbolic use. Carol Weiss (1977) introduced the no- 
tion of conceptual use, which highlights its enlightenment 
function. This is when research influences how policymak- 


ers and practitioners think about issues, problems, or 
potential solutions. More recently, Weiss and colleagues 
(2005) wrote about imposed use. This refers to recent 
government initiatives that tie funding to the adoption 
of evidence-based programs (e.g., Safe and Drug Free 
Schools legislation in the 1990s and 2000s). Lastly, pro- 
cess use refers to what practitioners learn from partici- 
pating in the production of research, as opposed to how 
they apply or learn from research findings. 


The Social Ecology of Research Use 

Research use unfolds within a social ecology of relation- 
ships, organizational settings, and political and policy 
contexts (Coburn, Honig, & Stein, 2009; Nutley, et al., 
2007; Tseng, 2008). Prior knowledge utilization studies 
tended to examine research use at the individual level, 
focusing on decision-makers’ preferences and beliefs 
about research and their self-reported uses or likelihood 
of using research. A narrow focus on cognitive, affective, 
and motivational processes, however, paints a decontex- 
tualized picture of how research is acquired, interpreted, 
and used. The science of research use needs to encom- 
pass the ways in which social, political, and economic 
forces affect individual and group processes. 

Relationships. Policy and practice occur within 
social systems-webs of relationships and interactions 
with peers and intermediaries. It is not surprising, then, 
that relationships are important pathways by which 
policymakers and practitioners acquire, interpret, and 
use research. Indeed social capital theory suggests that 
individuals and groups access resources, including in- 
formation, through their social ties (Daly & Finnigan, 
in press). Trust is key. When confronted with questions 
about a program or reform, agency administrators ask 
their trusted peers working in like positions, serving the 
same populations, and working under comparable condi- 
tions. As mentioned earlier, relationships with intermedi- 
aries are also vital-agencies look to technical assistance 
providers, professional associations, and consultants, and 
legislators rely on interest groups, to distill research find- 
ings into implications for their work. 

Organizational Context. Organizational capacity, 
culture, and structure also shape research use. Agencies 
are facing demands to become more evidence-based, but 
for many of them it is has not been part of their histori- 
cal charge. School district central offices, for example, 
are expected to serve as instructional leaders to improve 
teaching and learning, but they have historically been set 
up to serve more bureaucratic functions such as manag- 
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ing facilities, school buses, and purchasing and as admin- 
istrative pass-throughs for federal and state programs 
(Honig, 2008). The ability to acquire, interpret, and use 
research requires adequate staff capacity, time, and 
expertise, as well as conducive organizational cultures 
and routines (Moynihan & Landuyt, 2009). Organizational 
structure is also important. Some observers have sug- 
gested that bureaucratic silos may obstruct the use of 
research, particularly when departments analyzing data 
for accountability purposes have limited interactions with 
those making program decisions. In contrast, individu- 
als who work together in task force, committee, and 
team meetings and who have opportunities to informally 
converse can develop shared understandings about what 
constitutes high-quality evidence and the uses of it 
(Coburn, et al. , 2009). 

Political and Policy Contexts. Advocates of evi- 
dence-based policy often bemoan the role of politics, 
values, and ideology in policy; they suggest that using 
research should produce more rational decisions. Carol 
Weiss (2000), however, argues the opposite-that far 
from settling political debates, research is often used 
as a weapon in them. Nisbet and Scheufele (2009, p. 

1768) too suggest that “no matter how accurately com- 
municated and understood the science, policy decisions 
cannot be separated from values, political contexts, and 
necessary trade-offs between costs, benefits, and risks.” 
Rather than viewing politics as a nuisance to be set aside, 
it behooves us to increase our understanding of how the 
political and policy process works and how it influences 
research acquisition, interpretation, and use. In addition, 
are there political conditions or policy contexts in which 
research is more likely to be used in particular ways? This 
stronger knowledge would allow research producers to 
be better prepared for the ways their work may be used. 
Those who want their work to have greater impact may 
also learn lessons about how and when advocates, politi- 
cians, and agency leaders are able to leverage research 
to further policy goals. 

Toward Stronger Theory 
and Empirical Evidence: 

A Multidisciplinary and Mixed-Methods Field 

At the William T. Grant Foundation, we have sought to 
build stronger theory and empirical evidence on when, 
how, and under what conditions research is used. We have 
commissioned conceptual and exploratory work 
(Davies & Nutley, 2008; Nelson et al., 2009) and issued an- 


nual requests for proposals. We have funded 15 research 
studies and expect to fund more in the coming years. 

Current scholarship builds on early knowledge 
utilization studies and more recent work on the use of 
research, data, and evidence-based programs. In addi- 
tion, investigators draw on work from various fields that 
illuminates the social ecology of policy and practice-how 
government operates, the decision-making processes, 
and the central players who influence what occurs. This 
work comes from political science, public administration, 
and sociology on the policy process, deliberation, policy 
implementation, and organizational learning. Scholars 
also draw on theories and methods on social networks, 
diffusion of innovations, and cultural exchange in order 
to examine the social dynamics involved in acquiring and 
interpreting research. 

Prior work on research use often consisted of 
post-hoc case studies examining past instances when 
research seemed to have been used, and interviews or 
surveys asking decision-makers about their attitudes, 
uses, or likelihood of using research. In our funding, we 
have encouraged prospective, longitudinal, and mixed- 
methods studies (Small, 2011; Yoshikawa, Kalil, Weisner, 
& Way, 2008). Multiple sources of data are collected and 
analyzed in order to deepen understanding and validate 
findings across data sources. For example, investigators 
have observed meetings at which research is used to 
make decisions, interviewed participants in the meet- 
ings regarding their perceptions of what occurred and 
the research discussed, and coded the documents that 
went into and came out of the meetings. This mixed- 
methods work is an advance over single-method studies 
that relied on self-reports because it allows investiga- 
tors to validate interview findings against observations 
and reviews of pertinent documents. Prospective data 
collection also helps mitigate the problems of recall 
bias and post-hoc reconstruction of events. Social net- 
work analysis is employed to map the social system and 
how research flows through it. 

Many of these projects are comparative case 
studies (Yin, 2003). Intensive data collection in a small 
number of sites allows investigators to closely observe 
and refine the concept of research use-seeing what it 
looks like and how it unfolds. Sites are selected based 
on theory about what produces variation in the use of 
research. For example, Goertz and her colleagues are 
examining state education agencies; they hypothesize 
that research acquisition and use are stronger in agencies 
with greater internal collaboration across units and those 


Social Policy Report V26 #2 


8 


The Uses of Research in Policy and Practice 


with greater access to local intermediary organizations 
(Massell & Goertz, 2012). She has thus sampled agen- 
cies that vary on these two dimensions and will examine 
whether her hypotheses bear out in the findings. 

The projects we are supporting are still in the field, 
but are starting to produce findings. Below I discuss early 
findings from three of these studies. 


School Districts’ Use of Research 

Two of these studies are in education, which has been 
on a steady march toward accountability in the past few 
decades. Federal policies, including No Child Left Be- 
hind (NCLB), have created unprecedented stakes around 
school districts’ use of evidence (Honig & Coburn, 2008). 
NCLB required districts to track and report student and 
school test scores and use “scientifically based research” 
in a range of decisions about school improvement. It also 
created a high-stakes accountability environment around 
evidence use by initiating a series of sanctions for schools 
that do not make Adequate Yearly Progress toward im- 
proving student achievement. Two years of inadequate 
progress triggers an improvement phase, which is fol- 
lowed by corrective action. Continued failure results in 
restructuring, which requires changes that can include 
staff reconstitution, charter school conversion, state 
operation, or privatization. 

It is within this policy context that Daly and 
Finnigan (2011), and Asen and Gurke (2011) are studying 
school districts-a crucial group of research users. The 
former are focusing on district administrators and school 
staff and the latter on school boards. 

District Administrators and School Staff. Finni- 
gan, Daly and Che (2011) are conducting a four-year 
longitudinal study of research use in two urban school 
districts in which a large and increasing number of 
schools have been placed in corrective action. Daly and 
Finnigan are mapping the social networks of teachers, 
school administrators, and district staff to understand 
with whom they communicate regularly, who they ap- 
proach for information about school improvement, and 
who they trust as sources of information. They are also 
conducting in-depth case studies with a subsample of 
schools; these include interviews with school and dis- 
trict staff, document reviews, and observations of meet- 
ings pertaining to school improvement. 

One of their goals is elucidating how practitioners 
define and interpret research evidence. Through inter- 
views, they are finding that educators in these districts 
hold a surprisingly narrow definition of evidence in which 


it is equivalent to student test scores. Many do not 
distinguish between the terms evidence and research 
evidence. As one educator bluntly stated, “Evidence is 
just test scores-that’s the bottom line.” 

When some staff spoke more broadly about re- 
search, they placed a premium on evidence of what 
would work in their local contexts and expressed doubt 
that what works in one place would work in others. Their 
findings echo those of Nelson and colleagues (2009), 
who found that a broad range of education stakeholders 
valued research conducted with local data or in sites that 
are similar to theirs in terms of size, demographics, and 
urban versus rural locale. 

Concerns about trusting research also emerged. 
Practitioners in their study overwhelmingly believed that 
research and evidence could be, and often are, manipu- 
lated to make a point. In essence, they believed that re- 
search is often used to buttress political agendas (Nutley, 
et al. , 2007; Weiss, et al. , 2005). As one educator stated, 
“You know, you can find research to support 
anything. The problems we have in our soci- 
ety today. . . . People are now using research 
to say that all the problems are the teacher, 
and if you can correct the teacher, all our 
problems go away, which is ridiculous. . . . 

The point is research can be slanted to sup- 
port many different viewpoints. It doesn’t 
mean it’s correct” (Daly & Finnigan, 2011). 
Relationships emerge as key conduits for acquir- 
ing and using research evidence. Teachers saw their 
principals as important sources of evidence from outside 
the school or from other classrooms. This was particu- 
larly true in the lower performing schools. Principals, in 
turn, seek to learn about evidence and research- based 
ideas from district staff. This suggests that whether 
teachers have strong ties to principals-and principals 
with key district administrators— influences whether 
research- based ideas get into schools. If this hypothesis 
bears out in further testing, it suggests that principals 
with stronger relationships to their districts and to their 
teachers are better positioned to access and make use of 
research, and that interventions to strengthen these ties 
may improve the diffusion of research. 

Schools Boards. Asen and Gurke are communica- 
tions scholars examining research use by school boards 
(Asen, Gurke, Solomon, Conners, & Gumm, 2011; Asen, 
Gurke, Conners, Solomon, & Gumm, 2012). Gurke is also 
a former school board member who now advises school 
boards on governance issues (Gurke, Asen, Conners, Solo- 
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mon, & Gumm, 2011). Their study examines how research 
and other types of evidence are interpreted and used as 
school boards communicate their positions, explain their 
reasoning, and seek to advance their positions while op- 
posing others. The project is being conducted in three 
school districts in Wisconsin. Data have been collected 
via observations and recordings of about 250 school board 
meetings including regular meetings, work sessions, com- 
mittee meetings, and public hearings over two years. In 
semi-structured interviews, they asked board members, 
superintendents, assistant superintendents, and staff 
about the boards’ decision-making processes, the infor- 
mation used in those processes, and their perceptions of 
the meaning and relevance of research for their districts. 
Lastly, they reviewed various district documents, media 
reports, and blogs related to 
the school board meetings. 

They first employed 
an observational approach 
to identify the different 
types of evidence used by 
school boards (Asen et al. , 

2011; Asen et al., 2012). 

Consistent with other stud- 
ies, they found that the 
school boards drew upon 
a broad array of evidence 
types. Research was used 
very infrequently. Examples 
were used most often, fol- 
lowed by experience, data, 
testimony, then research, 
and lastly law/policy (see 
definitions of these evidence 
types in Table 1 ). 

Their work shows that 
when research was used, it 
was not in the way envi- 
sioned in No Child Left Behind (Asen et al., 2011). Im- 
plicit in NCLB is the notion that research should be used 
in conceptual and instrumental ways— to better under- 
stand problems and make more sound decisions. Instead, 
Asen and his colleagues find ample indications of politi- 
cal uses. Most references to research consisted of brief, 
general statements of “research says.” One presenter, for 
example, argued at a board meeting that “research has 
shown that hunger can actually explain 27 percent of the 
differences in aggressive behavior among children” (Asen 
et al., 2011, p. 204). No additional information was pro- 


vided about the research, and it was not discussed again. 
The purpose of evoking research in these situations was 
not merely to report information. Rather research was 
raised because it strengthened the speaker’s argument 
for a particular course of action (in the above case, of- 
fering more nutritious school meals). Evoking research 
also bolstered a speaker’s credibility and authority. 

Vague references to research may mask the real 
points of contention. In one meeting, for example, two 
board members disagreed about what research showed 
about early education programs. Because their refer- 
ences to research remained general and neither engaged 
with each other’s research, it is not clear that they were 
discussing the same research. Instead, the debate about 
research obscured their underlying disagreement, which 

related to conflicting val- 
ues about whether commu- 
nities should pay for early 
education for low-income 
families. Perhaps propheti- 
cally, a third board member 
commented that “at the 
end of the day, you’re both 
going to be able to find 
studies that counter the 
other ones” (Asen et al., 
2012, p. 25). 

Proponents of 
evidence-based policy 
often argue that research 
should inject better 
information and more 
rationality into the policy 
process. These studies 
and others suggest a more 
complicated relationship 
between research, poli- 
tics, and policy (Honig & 
Coburn, 2008). Asen and Gurke’s study illustrates the 
ways research is used in political debates, while Daly 
and Finnigan’s work suggests that the high stakes asso- 
ciated with test scores may create unintended negative 
consequences, such as narrow definitions and distrust 
of research. Nelson and colleagues (2009, p. 24) simi- 
larly found a common belief among various education 
stakeholders that “research could be found to support 
any point of view and was therefore of little valid use.” 
In Spin Cycle, Henig (2008) documents how advocates 
on both sides of the charter school debate marshaled 


Table 1. 


Definitions of Evidence Types 

Used by School Boards (Asen et al., 2011) 


Term 

Definition 

Research 

Empirical findings derived from systematic 
analysis of information, guided by purposeful 
research questions and methods 

Experience 

First-hand knowledge, skill, or perspective de- 
rived from direct observation of or participa- 
tion in events or activities 

Testimony 

Representing through quotation or paraphrase 
the perspective of an individual or group 

Data 

Measurable quantitative or qualitative infor- 
mation systematically collected to describe a 
set of conditions or trends 

Example 

A specific case or incident used to illustrate 
typical or exceptional characteristics of a 
topic or issue 

Law/Policy 

Rules and regulations that permit or prohibit 
particular actions, behaviors, or programs 
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their research evidence and sometimes their distinct in- 
terpretations of the same studies to sway public opinion 
and policymakers. Far from settling political debate, 
research seems to become part of the fodder. 


Evidence-based Programs and Child Welfare Agencies 

There is widespread concern that despite a growing body 
of evidence-based programs, these programs fail to be 
adopted and implemented on a large scale. The National 
Research Council’s (2009) report on Preventing Mental, 
Emotional, and Behavioral Disorders Among Young People 
recommends that federal and state agencies prioritize 
the use of evidence-based programs and promote ongo- 
ing rigorous evaluation of them. There are currently six 
federal initiatives relevant to children and youth that tie 
program funding to evidence of effectiveness (Coalition 
for Evidence-based Policy, 2012). These initiatives span 
home visiting, teen pregnancy, education, employment, 
and youth programs. 

Despite the ascendance of the evidence-based 
practice (EBP) movement, relatively little attention is 
paid to how agencies define and appraise EBPs even 
though they are the would-be adopters and implemented 
of programs. Among researchers EBPs are usually de- 
fined as those with evidence of demonstrated impact in 
randomized controlled trials (Flay et al. , 2005; National 
Research Council, 2009). While there is not unanimity in 
researchers’ definitions, differences are often about the 
appropriate level of scientific rigor and evidence under- 
girding claims about effectiveness (Guerra et al., 2011). 

Palinkas and colleagues (2011a, 2011b) are study- 
ing how agency leaders and managers define, acquire, 
interpret, and use research when they consider adopting 
and implementing an evidence-based program. The study 
builds on a randomized controlled trial in 40 California 
and 11 Ohio counties. The experiment tests whether 
Community Development Teams lead to stronger imple- 
mentation of Multi-dimensional Treatment Foster Care 
(MTFC) by mental health, child welfare, and probation 
agencies as compared to control sites that obtain techni- 
cal assistance but are not involved with the Community 
Development Teams. MTFC is an evidence-based program 
for youth with emotional and behavioral problems that 
has been found effective in randomized controlled trials. 

Using interviews and focus groups, Palinkas and his 
colleagues (2011a) find that practitioners hold widely 
varying definitions of evidence-based practice-none of 
which mentioned validation through randomized con- 


trolled trials. They said evidence-based practices were 
practices that have been studied, have been applied to 
different populations, come with extensive training manu- 
als or curricula, have been around for a long time, have 
been monitored and tracked carefully, require rigorous 
training and fidelity to curriculum, have published out- 
come data, are found on lists of EBP practices, or change 
a client’s behavior and way of thinking. While some of 
the definitions refer to research, others do not; and none 
strongly matched researchers’ definitions. 

Palinkas and colleagues (2011a) also used the quali- 
tative data to develop quantitative measures to assess 
practitioners’ sources, evaluations, and uses of evidence. 
His measures assess these processes for individual prac- 
titioners and for group consensus. They find that system 
leaders evaluate research evidence primarily for its rel- 
evance to their local circumstances and implementation 
concerns. This is indicated by how well the study’s popu- 
lation matches their population, the program’s effects in 
counties with similar demographics as theirs, how much 
the program costs to implement, and how much time is 
required to train staff. System leaders use research to 
support decisions to adopt a program, compare informa- 
tion from experts or community members, determine a 
program’s potential harm, decide how much adaptation is 
necessary for their circumstances, and find programs that 
meet the needs of their populations. 

While researchers have focused largely on ques- 
tions of internal validity, the questions at the forefront of 
practitioners’ minds are often questions of external valid- 
ity and implementation. This is not to say that evidence 
of internal validity is not important to them, but it does 
suggest that researchers’ significant progress shoring up 
internal validity claims about program effectiveness has 
not met the questions at the top of practitioners’ minds. 
Moreover, there is currently little empirical evidence on 
their questions. 

Similar to Daly and Finnigan (in press), Palinkas 
and his team (2011b) have been interested in the role 
of relationships in practitioners’ acquisition of research. 
Their findings suggest the importance of opinion leaders, 
whose positions in a social network enable them to influ- 
ence others. Palinkas (2011b) used social network analy- 
ses to identify the individuals who were seen as sources 
of information and advice about evidence-based programs 
and innovative programs more generally. Longitudinal 
analyses two years later demonstrated that these opinion 
leaders worked in counties that made greater progress im- 
plementing MTFC. In semi-structured interviews, agency 
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leaders and managers elaborated on their reliance on 
other counties and intermediary organizations for in- 
formation about funding, staffing, and clients. Small, 
rural counties were more isolated and relied to a greater 
extent on other counties for information and advice. 

Like many EBPs, MTFC requires collaboration across 
agencies (in this case, local mental health, child welfare, 
and probation departments) to be successfully implement- 
ed. Palinkas seeks to understand what shapes consensus 
on evidence use and the types of interactions between 
researchers, practitioners, and intermediaries that fa- 
cilitate the use of research evidence. Because his work is 
embedded within a randomized controlled trial, he will be 
able to test whether the Community Development Teams 
create stronger interactions between researchers, practi- 
tioners, and intermediaries and whether those interactions 
are associated with greater progress implementing MTFC. 

Drawing Early Lessons 
This is still a relatively young field of 
study, and it needs to develop a more 
robust evidence base. If findings bear 
out in further study, however, the les- 
sons could be sobering. 

We can be more strategic in iden- 
tifying the would-be users of research 
and increasing our understanding of 
their work. Child development re- 
searchers often seek to inform federal 
policymakers and frontline practitio- 
ners, but agency leaders and managers 
at the state and local levels should not 
be overlooked. In child welfare, juve- 
nile justice, health, mental health, and 
education, these are the organizations 
and people who shape the reforms, 
programs, curricula, and staff devel- 
opment that influence youth’s daily 
settings and lives. 

Early lessons also suggest that 
the research community’s valiant ef- 
forts to improve rigor— though vitally 
important-are insufficient. In the 
area of evidence-based programs, 
the research community has come a 
long way in strengthening standards 
of evidence on what works, but little progress has been 
made on critical questions for the would-be adopters of 
programs. Their questions about what it takes to imple- 


ment programs and whether they would be effective with 
different populations, under different operating condi- 
tions, and in different contexts have been studied too 
infrequently. Building a robust evidence base on these 
questions should be a priority. 

Getting the research “right” and communicating it 
clearly can only get us part-way down the road to research 
use. Practitioners and policymakers always need to inter- 
pret the meaning of research and its implications for their 
particular problems and circumstances. Thus, we do not 
simply face a communications problem of better convey- 
ing research; nor is it merely a dissemination problem of 
better distributing research. Translation is critical, and we 
should reflect more intentionally on who makes for the 
best translators and how to create productive contexts for 
translation. Some people argue that researchers should be 
the translators because of their expertise drawing con- 
clusions based on research design and knowledge of how 
findings fit with the broader literature. 
Others contend that intermediaries are 
better positioned as translators because 
their jobs focus on influencing policy 
and practice, and they already have 
trusted relationships with decision-mak- 
ers. In either case, people need pro- 
ductive contexts for deliberating over 
research. As important as getting the 
right people in the room is creating the 
conditions for back-and-forth discus- 
sions about the research and its impli- 
cations for particular circumstances. 

A third option is for interme- 
diary organizations to be relation- 
ship brokers, bringing researchers 
and decision-makers together to 
focus on core problems of practice 
or policy. This approach capitalizes 
on the expertise of intermediar- 
ies and researchers without asking 
them to stretch beyond their means. 
Decision-makers will always have 
questions about how research applies 
to their local circumstances, and 
there is limited research evidence on 
many of those questions. However, 
researchers and decisions-makers 
can jointly discern between more 
or less plausible conclusions from existing evidence. 
Furthermore, researchers’ sustained engagement with 
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decision-makers can drive them toward more relevant 
research agendas. Policymakers, in turn, could start to 
see researchers as trusted sources and develop a bet- 
ter feel for what research can and can’t provide. 

What it means to use research and, more specifi- 
cally, what productive research use entails is still wide 
open for debate, and it is a debate worth having-not 
simply within the research community but with those we 
hope will put the research to use. Proponents of evi- 
dence-based policy and practice often aspire to rational, 
instrumental uses of research. Weiss (1977), however, ar- 
gues that research more often achieves influence through 
a slow, diffuse process in which it seeps into policy and 
shifts how people think about problems and orient them- 
selves to issues. This conceptual use of research is harder 
to track, but it probably better matches real-world 
complexities. Like all of us, policymakers and practitio- 
ners integrate new information into their existing store 
of knowledge, and when research is used, it has already 
been transformed into implications and inferences (Ken- 
nedy, 1984; Nutley et al. , 2007). Policy and practice 
work is also complicated-influenced by different types 
of evidence and negotiation of political and fiscal condi- 
tions-and thus it is the rare occurrence when research 
plays the defining role in a particular decision. 

This brings us to the political uses of research, 
which are hard to judge in absolute terms as either good 
or bad. Policymakers and practitioners need to justify 
their stances and using research to “add weight and heft 
to a position” that is supported by a body of research 
is not necessarily a negative event (Weiss, 2000, p. 

299). Intentional distortions of research, however, are 
discomforting, as are passing evocations of “research 
says” that muddy what research actually says and mask 
the real points of disagreement. The general distrust 
and cynicism of research expressed by would-be users 
are also troubling. Gurke et al. (2011) suggests that 
these issues can be mitigated if school boards identify 
the specifics of the research they are discussing and 
acknowledge their own values and that underlying the 
research. Nisbet and Scheufele (2009) too argue that 
public deliberation of research should acknowledge 
stakeholders’ diverse values and goals. 

Relationships are emerging as key conduits for re- 
search, interpretation, and use. Policymakers and practi- 
tioners rely on trusted peers and intermediaries. Rather 
than pursuing broad-based dissemination efforts, there 
may be value in understanding the existing social system 
and capitalizing on it. Dearing and Kreuter (2010) lay out 


a strategy that involves (1) clearly delineating the sector 
you seek to influence, (2) collecting data to map the social 
structure within that sector and identify its opinion leaders 
(i.e., the states or localities others look to for innovative 
ideas, intermediary organizations that are seen as trusted 
sources of information), and (3) recruiting those opinion 
leaders to help with dissemination efforts. 

Building partnerships with policymakers and prac- 
titioners is another strategy. In education, there is an 
increasing number of institutional partnerships between 
researchers and school districts. Many are modeled 
after the long-standing Consortium on Chicago School 
Research, in which researchers study the core problems 
facing the district and seek to build the district’s capacity 
to improve practice (Roderick, Easton, & Sebring, 2009). 
The Institute for Education Sciences’ new contracts for 
Regional Educational Laboratories support similar alli- 
ances with multiple local and state education agencies. 
Bryk (2009) and Donovan (201 1 ) also argue for partner- 
ships focusing on core problems of practice, but envision 
researchers’ roles as co-developing and testing innova- 
tions with practitioners. With the intensive, iterative 
engagement of research and practice partners, these 
partnerships models may yield broader lessons for the 
field about how to produce rigorous, usable, and timely 
research that is constructively used. 

I began this paper with the simple idea that we 
need to better understand the would-be users of re- 
search. I would like to end by encouraging us to recast 
our gaze back on our research community. Understand- 
ing users, taking on translation roles or working with 
translators, and building partnerships require shifts in 
our practices, incentive systems, and expertise. Relation- 
ships are resource-intensive. It takes time and money to 
build relationships, establish trust, and develop shared 
commitments-and more of the same to maintain them 
(Donovan, 201 1 ). As we move forward, we will need to 
critically reflect on how to bring our work and institutions 
in line with our aspirations for research use. ■ 
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'he Uses of Research 

Karen Cadigan 

Minnesota Office of Early Learning 

in Policy 

and Practice 


his critically impor- 
tant topic-the use of 
research in policy and 
practice— is one that 
should be on the mind 
of every researcher 
interested in applied sciences and 
everyone, researcher or not, who be- 
lieves that social science has a role 
in making the world a better place. 

In this article, Tseng makes some 
dramatic claims. Without strong 
theory and empirical evidence, she 
notes, “it seems likely that the hope 
for evidence-based policy and prac- 
tice will unravel-another fad tried 
and failed.” She asserts that “schol- 
ars’ attempts to make research more 
usable and to communicate it more 
effectively run the risk of missing 
the mark” without a better under- 
standing about “the other side” of 
research use. 

These assertions are spot on; 
researchers, research institutions, 
and those that provide resources for 
both should heed them urgently. 

This commentary is focused 
on the uses of research in a state 
level policy sphere, that Tseng ac- 
curately dubs an “oft-missed but 
critical group of researcher users.” 
This context is especially important 
because of the significant influence 
states have not only on other levels 


of policy, both local and federal, but 
also on practice. For example, local 
school district policies and practices 
are impacted by a state’s policies 
on whether or not kindergarten is 
mandatory, on whether children 
with risk factors are included in Part 
C eligibility, and on what the state’s 
testing requirements are before 
high school graduation. At a more 
macro level, states long have been 
understood to be the “laboratories 
for democracy. ” As then Justice 
Louis Brandeis described, “a single 
courageous State may, if its citi- 
zens choose, serve as a laboratory; 
and try novel social and economic 
experiments without risk to the 
rest of the country” (New State Ice 
Co. v. Liebmann, 1932). In choos- 
ing a single one of those concentric 
rings, then, from Bronfenbrenner’s 
ecological model (1979), the role of 
the state policy in the macro system 
is critical for the impact it has on 
children’s development. 

Bogenschneider and Corbett 
(2010)-who have spent significant 
time both practicing and studying 
the practice of connecting research 
to state-level policymaking-posit 
a community dissonance theory to 
explain the gap, which I find the 
most satisfactory framework to date. 
This theory brings together Kingdon’s 


(1995) idea of open policy windows 
and Caplan’s (1979) two communities 
description. That is, even on the oc- 
casion when a policy problem is well 
matched with research, communica- 
tion between knowledge producers 
(researchers) and knowledge consum- 
ers (policymakers) is difficult. These 
difficulties arise from a complex 
mismatch of culture between the two 
communities. In other words, there 
are challenges both on the supply 
and demand sides of research use in 
policymaking, and these challenges 
are deeply rooted on each side. 

A peculiarity about state 
policymaking that sets it apart, I 
believe, from the general “policy 
and practice” settings that Tseng 
describes, is that in the context of 
the two communities theory, the dis- 
tance that researchers and state pol- 
icymakers have to reach one another 
is a long, long way. In other cases, 
the research producing and research 
using communities are more proxi- 
mal, with some shared language and 
other cultural artifacts that allow for 
improved research use. For example, 
the “distance” between local school 
boards and research on early reading 
intervention can likely be bridged 
with some improved translation of 
research, presenting to important 
decision makers in a way that makes 
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it relevant for their context, show- 
ing cost-benefit analyses, and using 
other tools that researchers have to 
make research compelling. Practi- 
tioners and even local policymak- 
ers may be in differing time zones 
from researchers-hard to reach, but 
reachable. Researchers and state 
level policymakers are communities 
on different planets. 

Six months ago I moved to the 
other planet (state agency leader- 
ship) from University life, and my 
perspective on the gap has changed 
dramatically. As those who work at 
policy education, especially those 
other 26 states that are Family 
Impact Seminar sites, know there 
are supply-side challenges in the 
practice and study of connecting 
research to policymaking (e.g., Bo- 
genschneider, Little, Ooms, Benning, 
Cadigan & Corbet, 2012). University 
promotion and tenure systems, for 
example, largely do not put em- 
phasis on translational activities or 
allow for the time commitment that 
is required in building relationships 
with policy makers. Many public and 
private grant makers stay far away 
from funding outreach activities or 
research that includes policymaking, 
often with the (incorrect) assump- 
tion that any thing relating to policy- 
making is political or advocacy based 
on the part of the researcher. “The 
only thing we’re advocates for,” I 
used to say, “is the use of research 
in policymaking.” 

Now on “the other side,” I’m 
aware that my understanding of poli- 
cymakers’ needs was like the shad- 
ows in Plato’s cave, naively limited 
in its view by partial understanding 
of the context. I certainly don’t 
claim to be much closer to fully 
understanding the context in which 
I now work. It still feels like being 
on a field trip in a foreign land. This 


decision-making context is one that 
Tseng succinctly summarizes as “a 
complicated relationship between 
research, politics, and policy.” I 
crave an understandable, compre- 
hensive, and useful description of 
why using research in a policy setting 
is so difficult, and look forward to 
learning a better way, informed by 
the research agenda Tseng suggests. 

The good news is that there 
have been many instances, more 
than I expected actually, in which 
I made a decision-some small, 
some big-based on my knowledge 
of research or my ability to access 
it quickly. The occasions where 
my knowledge and connections to 
research/ researchers paved the way 
for policy to be informed by research 
have been those with a common 
recipe of clear problem definition, 
open policy window, and personal 
power. The decisions “best” in- 
formed by research have been those 
with a discrete number of options, 
where a choice needed to be made, 
in which I was the only or the lead 
decision maker, and in which there 
was no politically charged context 
of note. In my experience thus far, 
whether a final policy direction has 
been informed by research or not, a 
common denominator has been the 
speed of turn around time. The pace 
with which decisions are made, the 
volume of the work, the enumerable 
priorities in state level leadership 
are all beyond what I could have 
ever imagined. This is one of the ma- 
jor differences between the policy 
making and research producing 
worlds. An unfortunate result of this 
difference has been that on more 
than one occasion I sought out re- 
search to inform my decision-making 
and found my research colleagues 
and my beloved research institution 
to be something closer to irrelevant 


than essential. In most cases this was 
simply because the response came 
weeks, not hours later. Sometimes 
the relevancy challenge was that the 
response was a set of ideas, not an 
actual answer. This is a hard thing to 
admit, but I feel some responsibility 
to be honest about it because I be- 
lieve it provides an accurate descrip- 
tion of how difficult it is to bridge 
the gap between research and state 
level policy making, even when the 
demand side is set up nearly as well 
as it could be. 

These issues of decision-mak- 
ing volume, pace, power, and policy 
windows all lead me to conclude 
that here’s what it comes down 
to: infusing even the most useable 
research into the policy world isn’t 
enough. Infusing researchers into 
the policy world is needed, too. We 
need more expatriates. 

The call for more expats, is 
supported by several findings that 
Tseng notes: “relationships and trust 
are important conduits for acquiring 
and using research evidence” and 
that “research use unfolds within a 
social ecology of relationships, orga- 
nizational settings, and political and 
policy contexts.” Especially because 
of the important role of relationships 
and social contexts in acquisition and 
use of research, having someone with 
research knowledge in a policymaker 
or implementer role can have signifi- 
cant ripple effects. I expect that one 
of the marks of my tenure in state 
government that will last after my 
presence will be my hiring of those 
who have research experience and 
teaching others how to find and ap- 
ply research to their work. 

The focus of bridging the gap 
between research and policy must 
include not only the supply side 
(where researchers spend all their 
time and, as Tseng notes, where the 


Social Policy Report V26 #2 


18 


The Uses of Research in Policy and Practice 


study of these issues has focused), 
but also on the demand side. I 
maintain hope for and commit- 
ment to significant improvement 
in evidence-based policy making. 
While researchers’ study of the gap 
is important for this improvement, 
it is also the case that more re- 
searchers should think beyond just 
understanding policymakers and 
should think about becoming them. 
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Commentary 


John Q. Easton 

Institute of Education Sciences 


ince becoming direc- 
tor of the Institute of 
Education Sciences 
(IES) in June 2009, I 
have added my voice 
to the chorus call- 
ing for more relevant and useful 
educational research to help im- 
prove teaching and learning and the 
systems that develop and support 
them. As the federal government’s 
home for education research, evalu- 
ation, statistics and assessment, 

IES influences how some education 
research is planned, conducted, and 
to a lesser extent, used. One of our 
main strategies to promote usability 
and relevance has been to encour- 
age partnerships among researchers, 
practitioners and policy makers. In 
the fall of 2010, our advisory board, 


the National Board of Education 
Sciences, approved a new set of re- 
search priorities for IES that includes 
the following statements: 

The work of the Institute is also 
grounded in the principle that effec- 
tive education research must address 
the interests and needs of education 
practitioners and policymakers, as 
well as students, parents and com- 
munity members. To this end, the 
Institute will encourage research- 
ers to develop partnerships with 
stakeholder groups to advance the 
relevance of the Institute’s work, the 
accessibility of its reports, and the 
usability of its findings for the day-to- 
day work of education practitioners 
and policymakers (IES, 2010). 

We encourage these partner- 
ships in our grant making, and some- 


times we require them, as for ex- 
ample in a new research topic called 
Research-Practitioner Partnerships in 
Education Research (IES, 2012). One 
premise here is that practitioners are 
more apt to “take up” research find- 
ings that they’ve had a part in plan- 
ning and conducting. Another is that 
the researchers are highly tuned in to 
local needs, capacities and context. 

The IES emphasis on relevance 
and usability is influenced by my own 
previous experience at the Consor- 
tium on Chicago School Research 
(CCSR) at the University of Chi- 
cago where we worked as research, 
evaluation and analytic partners to 
the Chicago Public Schools as they 
assessed their needs and devel- 
oped and implemented improve- 
ment strategies (Roderick, Easton 
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& Sebring, 2009). In recent years, 
numerous similar researcher-school 
district partnerships have developed 
across the country in Baltimore, New 
York City, San Diego, and Kansas City, 
MO, for example. These, like CCSR, 
are predicated on the value of close 
relationships and high levels of trust 
between the researchers and leaders 
and staff in the school districts. 

Tseng’s paper provides a ser- 
vice to both researchers who want 
their work to help improve schools 
and to educational leaders seeking 
assistance in how to make better 
decisions about school improvement 
strategies and tactics. The conceptu- 
al framework helps build a common 
vocabulary and lays out topics that 
frequently confuse and frustrate 
both researchers and practitioners. 
The W. T. Grant Foundation’s invest- 
ment in building a knowledge base 
around the use of research is com- 
mendable, as the need is great. The 
fruits of this research can inform 
our work at IES as well as countless 
others concerned about this topic. It 
also provides stimulus to researchers 
to test many of their assumptions 
about the role of trust, relation- 
ships, social networks and local 
context, for example. 

Other scholars and practitio- 
ners across the country are engaged 
in related work. In a recent paper, 
Peurach and Glazer (2011) describe 
the process of replication or “scal- 
ing up” as a collaborative, long- 
term relationship between program 
developers (researchers) and schools 
or school systems (users), rather 


than as a linear progression from 
research to implementation. Using 
the development of Success for All 
as a case study, Peurach and Glazer 
describe how the researchers regu- 
larly redesigned their programs to 
accommodate local context and 
to meet the need for adaptability. 

At the same time, the users were 
constantly providing feedback. The 
replication or scale-up effort be- 
came a research endeavor in its own 
right as the knowledge gained in 
implementation was reincorporated 
in the program development. The 
use of research is actually part of a 
continuous improvement cycle. 

In a similar vein, Anthony Bryk 
and his colleagues at the Carnegie 
Foundation for the Advancement 
of teaching have launched a series 
of programs based on premises of 
improvement science, which bal- 
ances efforts aimed at developing 
and discovering “what works” with 
equally important efforts to under- 
stand “how to make it work” (Bryk, 
2009). The second step here-under- 
standing how to apply research find- 
ings in new situations— is especially 
critical given the great variability in 
context, needs, and capacity across 
different learning environments. 

This is a fundamental shift from the 
dominant “research to use” para- 
digm. Tseng’s paper and the W. T. 
Grant Foundation research initiative 
are helping to advance and under- 
stand this shift as they legitimize 
the dynamic relationships between 
researchers and users as an impor- 
tant and researchable topic. 
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Commentary 


Aletha C. Huston 

University of Texas at Austin 


T hough many scholars 
are committed to 
providing policymak- 
ers with research-based 
knowledge, the dearth 
of information about 
how that knowledge is (or is not) 
used is striking. After a long hiatus in 
the study of knowledge utilization, 
the WT Grant Foundation is making 
an important contribution by taking 
the scholarship several steps forward. 
In these comments, I note some of 
the most important points made by 
Tseng, then I offer some additional 
thoughts based on my own experi- 
ence (which is informed by, but not 
limited to research). 

The core question is: What does 
it mean to use research? Ambiguity 
about the answer to this question is 
one reason that researchers’ com- 
munications often miss the mark 
and why “uses” of research are so 
difficult to study. Because humans 
typically use their whole base of 
knowledge, often without distin- 
guishing the sources from which it 
is drawn, it is sometimes difficult or 
impossible to identify a particular 
knowledge source. This is particularly 
true for “conceptual” or indirect uses 
of research, in which evidence-based 
knowledge becomes part of a policy- 
maker’s broad store of information. 
How can we detect conceptual uses? 
Is it necessary for “users” to recog- 


nize that they are using research? 
Some of the research described here 
includes clever strategies for identi- 
fying use or non-use, but the prob- 
lem continues to be thorny. 

Tseng argues cogently for 
distinguishing policy from practice, 
with the further point that prac- 
titioners are often the “users” of 
research. I would frame the issue 
slightly differently, in line with 
Tseng’s other important point that 
mid-level managers (e.g., directors 
of state and local health depart- 
ments, school administrators, direc- 
tors of agencies overseeing early 
education and child care subsidy 
programs) are often the decision- 
makers about what practitioners do. 
Those managers are policy-makers as 
well as practitioners, making deci- 
sions about such matters as adopt- 
ing maternal-child health programs, 
educational curricula, and quality 
requirements for child care. I would 
add still another group of practitio- 
ners— individuals in the judicial sys- 
tem, including family court judges 
and professionals who deal with ju- 
venile offenders, many of whom are 
eager for more information about 
how to make good decisions. 

The research community now 
spends a great deal of time and 
effort trying to reach legislators, 
especially at the federal level-ef- 
fort that might be more effectively 


targeted to individuals in state and 
local executive departments and 
the judicial system. Reaching the 
mid-level managers and practitio- 
ners has the added advantage of 
infusing information more broadly 
through the social networks iden- 
tified in the studies of users. An 
administrator considering changing 
programs is likely to call on people 
in parallel positions to ask them 
what they have tried and how it has 
worked. Data and “research” may 
well come into the conversation, 
but the information is transmitted 
through trusted personal contacts. 

Some of Tseng’s points il- 
luminate the broader issue of the 
different “cultures” of scholars and 
practitioners, described so well by 
Shonkoff (2000). Practitioners define 
“research” much more broadly than 
is usual in the scholarly community. 
For example, one current policy 
effort in the early childhood field 
is getting integrated data systems 
that combine data on child care 
subsidies, child care use, Head Start, 
pre-kindergarten, and other pro- 
grams with the assumption that good 
information will follow. Most scholars 
would assume that “research” in- 
volves skilled analysis of those data 
systems once they are in place to 
examine particular questions. 

Practitioners and policymak- 
ers want information about external 
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validity and applicability to local 
conditions, requirements that often 
involve replications and exten- 
sions that are not rewarded or very 
interesting to researchers. When 
I present findings from our study 
of New Hope in Milwaukee to local 
audiences, for example, one of the 
first questions is whether a similar 
program has been tested in Texas. 
Replication across contexts is often 
touted in discussions of research, 
but is less often rewarded with fund- 
ing or scholarly recognition. 


One issue not addressed in 
this report is the threat to practi- 
tioners often posed by research and 
evaluation. The No Child Left Behind 
sanctions are a good example. If 
teachers perceive “research” as a 
threat to their competence, salary, 
or job security, they are unlikely to 
welcome the knowledge that it could 
provide. If administrators believe 
that “research” will be used to make 
decisions about funding or defunding 
for their programs, they may under- 
standably view it with alarm. Are 
there data on how research is used 
in such situations? 


Finally, I would like to see ef- 
forts to infuse systematic research 
methods into practice; that is, to 
have each practitioner ask for and 
think about data to inform their 
activities. Behavior analysts, for 
example, collect and use data on the 
individual’s behavior of interest to 
make decisions about the next steps 
in an intervention. Teachers can 
monitor children’s progress to de- 
termine when and how to introduce 
new material. Teaching practitioners 
and policymakers at all levels to 
respect and value systematic infor- 
mation in everyday decision-making 
could go a long way toward increas- 
ing “uses” of research. 
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